Papers with dense vector representations
Document-based Recommender System for Job Postings using Dense Representations (N18-3)
Copied to clipboard
| Challenge: | 45% of job posting traffic is driven by recommender systems for job postings . a large-scale job recommendation system is needed to detect similarity between job posting and item-to-item based recommendations. |
| Approach: | They propose to use dense vector representations to enhance a large-scale job recommendation system and rank job advertisements regarding similarity. |
| Outcome: | The proposed method increases the click-through rate on job recommendations by 8.0%. |
attr2vec: Jointly Learning Word and Contextual Attribute Embeddings with Factorization Machines (N18-1)
Copied to clipboard
| Challenge: | popular word embeddings are used to learn vector representations from the context of words. |
| Approach: | They propose a framework for jointly learning embeddings for words and contextual attributes based on factorization machines. |
| Outcome: | The proposed framework improves on a text classification task compared to learning embeddings independently. |
Discrete Cosine Transform as Universal Sentence Encoder (2021.acl-short)
Copied to clipboard
| Challenge: | Modern sentence encoders capture underlying linguistic characteristics of words . Discrete Cosine Transform (DCT) is an efficient alternative to averaging . |
| Approach: | They propose to use a Discrete Cosine Transform to generate universal sentence representations in different languages. |
| Outcome: | The proposed model captures the underlying syntactic characteristics of a given text without compromising practical efficiency. |
Condenser: a Pre-training Architecture for Dense Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Prior work fine-tunes deep LMs to encode text sequences into single dense vector representations, but dense encoders require a lot of data and sophisticated techniques to train and suffer in low data situations. |
| Approach: | They propose to pre-train Transformer language models (LMs) with a novel Transformer architecture, Condenser, where LM prediction CONditions on DENSE Representation. |
| Outcome: | The proposed model improves on various text retrieval and similarity tasks by large margins over standard LMs. |
BoolQuestions: Does Dense Retrieval Understand Boolean Logic in Language? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Dense retrieval systems focus on optimizing text embedding space while overlooking Boolean logic in language. |
| Approach: | They propose a task to investigate whether retrieval systems can comprehend Boolean logic in language. |
| Outcome: | The proposed method is based on a benchmark dataset covering complex queries containing basic Boolean logic and corresponding annotated passages. |
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing open-domain question answering systems are insufficient to capture deep semantic matching that goes beyond lexical overlaps. |
| Approach: | They propose a sample-efficient method to pretrain the paragraph encoder using an existing pretraining model instead of heuristically created pseudo question-paragraph pairs. |
| Outcome: | The proposed method outperforms a strong dense retrieval baseline that uses 6 times more computation for training. |
Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Word vector representations suffer from a monolingual bias due to the amount of data available across languages. |
| Approach: | They propose a technique for building language-independent vector representations of concepts which places multilinguality at its core while retaining explicit relationships between concepts. |
| Outcome: | The proposed representations outperform the state-of-the-art in multilingual and cross-lingual Semantic Word Similarity and Word Sense Disambiguation on low-resource languages. |
Making Fast Graph-based Algorithms with Graph Metric Embeddings (P19-1)
Copied to clipboard
| Challenge: | Graph measures, such as node distances, are inefficient to compute. |
| Approach: | They propose a way to learn graph embeddings by using vector operations instead of a graph structure. |
| Outcome: | The proposed method outperforms other graph embeddings on word similarity and word sense disambiguation tasks. |
Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings (2025.acl-long)
Copied to clipboard
| Challenge: | Text embedding models are used for various natural language processing tasks such as sentiment analysis, text clustering, and content-based information retrieval. |
| Approach: | They propose a synthesis framework that leverages large language models to generate diverse negative samples with varying levels of similarity with the query. |
| Outcome: | The proposed framework achieves state-of-the-art performance surpassing existing synthesis strategies with synthetic data and when combined with public retrieval datasets. |